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(54) Display widget interaction in embedded systems using child graphics contexts 



(57) A method and apparatus implementing a sep- 
arate child context for each applet (or similar element) 
of a browser. A described embodiment of the present 
invention provides one or more child contexts that cor- 
respond to elements in the HTML for a web page dis- 
played by a browser. For example, each applet execut- 
ed by the browser has a corresponding and separate 
child context Each child context has an associated 
memory buffer The browser also has a parent context, 
which each child context points to. When a graphic is 
displayed via a widget, the widget draws the graphic 
(such as a panel or a non -pressed button) in the child 
context of the applet and sets a "damage" flag in the 



child context. When the browser performs its main 
browser loop, it checks the status of the damaged flag 
for each element (including each applet). If the browser 
finds a damage flag that is set, this means that some- 
thing was written into the child buffer and that the parent 
buffer needs updating. In this case, the browser "pulls" 
the information from the child buffer into the parent buff- 
er, which is then used to update the display screen. Oth- 
er components, called reactive components, present 
special problems and are treated specially. Reactive 
components are drawn directly into both the child and 
parent contexts and buffers without waiting for the main 
browser loop. 
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Description 

FIELD OF THE INVENTION 

[0001 ] This application relates to a method and appa- 
ratus for a user interface control and, specifically, to a 
method and apparatus that allows for fast processing of 
reactive components in a user interface. 

BACKGROUND OF THE INVENTION 

[0002] Systems that have embedded programs, such 
as certain information appliances and consumer prod- 
ucts, often do not contain general purpose windowing 
software. In these systems, all graphics are generally 
done via calls to the built-in graphics routines, such as 
draw-line, draw rectangle, etc. In many conventional 
systems, graphics are drawn via interaction with the 
browser. Thus, if multiple applets are executing as dif- 
ferent threads in the browser, they will experience a lack 
of concurrency when performing graphics operations. 

SUMMARY OF THE INVENTION 

[0003] A described embodiment of the present inven- 
tion implements a separate child context for each applet 
(or similar browser element). A described embodiment 
of the present invention provides one or more child con- 
texts that correspond to elements in the HTML for a web 
page displayed by a browser. For example, each applet 
executed by the browser has a corresponding and sep- 
arate child context. Each child context has an associat- 
ed memory buffer. The browser also has a parent con- 
text, which each child context points to. When a graphic 
is displayed via a widget, the widget draws the graphic 
(such as a panel or a non-pressed button) in the child 
context of the applet and sets a 'damage" flag in the 
child context. When the browser performs its main 
browser loop, it checks the status of the damaged flag 
for each element (including each applet). If the browser 
finds a damage flag that is set, this means that some- 
thing was written into the child buffer and that the parent 
buffer needs updating. In this case, the browser "pulls" 
the information from the child buffer into the parent buff- 
er, which is then used to update the display screen. 
Thus, several separate threads in the JVM can be exe- 
cuting concurrently and updating graphics elements 
without interfering with each other. 
[0004] The invention extends the AGL (Applications 
graphics library) routines to include (at least) routines to 
set up, delete, read, and modify child contexts and to 
copy information from child contexts to parent contexts. 
The DWL (developer's widget library) routines are ex- 
tended to include (at least) a parameter identifying the 
context currently being worked on. The browser has 
been extended to implement the functionality of check- 
ing the damage flag of the context for each applet and 
pulling information from a buffer of a child context into 



the buffer of the parent context. 
[0005] Reactive components present special prob- 
lems. Reactive components are defined as components 
that must display interactive behavior. For example, 
5 when a user clicks on a button on the display screen, 
the button is momentarily displayed as "pressed" and 
soon thereafter displayed as "released." Such interac- 
tive behavior gives visual feedback to the user Exam- 
ples of reactive components include buttons, lists, 

io choice bars, etc. present special problems. When a 
widget, for example, a button, needs to show interactive 
behavior (such as going up and down when pressed), 
this behavior must occur very rapidly - often too rapidly 
to necessarily be picked up by a main browser loop, 

is which executes relatively infrequently. For example, in 
the described embodiment, the main browser loop exe- 
cutes approximately every 200 milliseconds (i.e., ap- 
proximately five times per second). In contrast, a "down" 
button press followed by an "up" takes approximately 50 

20 milliseconds. If the system were to wait for the browser 
loop, the button press would have to last much longer. 
[0006] In accordance with the purpose of the inven- 
tion, as embodied and broadly described herein, the in- 
vention relates to a method of displaying a reactive 

25 graphics element, comprising: sending a reactive 
graphics event, by a browser to a virtual execution en- 
vironment; providing a child context in the virtual execu- 
tion environment, corresponding to a parent context in 
the browser, where the child context corresponds to a 

30 current applet being executing by the browser; writing, 
by the virtual execution environment, a first graphic into 
a buffer of the child context and a buffer of the parent 
context, in accordance with the reactive graphics event, 
which will be displayed on a display screen; waiting a 

35 predetermined amount of time; writing, by the virtual ex- 
ecution environment, a second graphic into the buffer of 
the child context and the buffer of the parent context, in 
accordance with the reactive graphics event, which will 
be displayed on a display screen; and returning execu- 

40 tion control to the browser from the virtual execution en- 
' vironment. 

[0007] A fuller understanding of the invention will be- 
come apparent and appreciated by referring to the fol- 
lowing description and claims taken in conjunction with 
45 the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0008] The accompanying drawings, which are incor- 
so porated in and constitute a part of this specification, il- 
lustrate several embodiments of the invention and, to- 
gether with the description, serve to explain the princi- 
ples of the invention. 

[0009] Fig. 1 (a) is a block diagram of a data process- 
es ing system in accordance with one embodiment of the 
present invention. 

[001 0] Fig. 1 (b) is a block diagram of a data process- 
ing system in accordance with one embodiment of the 
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present invention. 

[0011] Fig. 2 shows an example of a set top box sys- 
tem, including a remote unit, and a video display. 
[001 2] Fig. 3(a) is a block diagram of parent and child 
context data structures in accordance with an embodi- 
ment of the present invention. 

[0013] Fig. 3(b) is a diagram showing the relation- 
ships between a browser, exemplary browser elements, 
and exemplary components of a specific browser ele- 
ment. 

[0014] Fig. 4 is a block diagram of an orphan data 
structure in accordance with an embodiment of the 
present invention. 

[0015] Fig. 5 is a block diagram showing a main 
browser loop and a JVM loop. 

[001 6] Fig. 6 is a flowchart showing details of the main 
browser loop, including refreshing the applets. 
[001 7] Fig. 7 is a flow chart showing details of refresh- 
ing the applets in the main browser loop. 
[001 8] Fig. 8 is a flow chart showing a main JVM loop 
for regular non-reactive components. 
[001 9] Fig. 9 is a flow chart showing a main JVM loop 
for reactive components, such as a button press. 
[0020] Fig. 10 shows further details of Figs. 8 and 9. 
[0021] Fig. 11 is a diagram of the architecture of one 
embodiment of the present invention. 
[0022] Figs. 1 2(a) and 1 2(b) are diagrams of, respec- 
tively, a pressed button and a released button. 
[0023] Fig. 1 3 is a flow chart for orphan context. 
[0024] Fig. 14 shows exemplary routines used to im- 
plement child and orphan contexts in an embodiment of 
the present invention. 

DETAILED DESCRIPTION OF PREFERRED 
EMBODIMENTS 

I. General Discussion 

[0025] Many conventional World Wide Web browsers 
are capable of executing Java applets. A Java applet is 
a computer program written in the Java programming 
language that is designed to execute within a World 
Wide Web browser. Such applets are often started when 
a user clicking on a button or link in a displayed Web 
page. Once started, an applet executes inside the 
browser, performing whatever tasks it has been pro- 
grammed to perform. 'Java* is a trademark of Sun Mi- 
crosystems, Inc. in the United States and other coun- 
tries. 

[0026] Fig. 1 (a) is a block diagram of a data process- 
ing system 100 in accordance with one embodiment of 
the present invention. Data processing system 100 can 
be, for example, a set top box 1 01 connected to a display 
device 130, such as, for example, a television set or 
some other appropriate display device. Data processing 
system 100 can also be, for example, any other appro- 
priat data processing system. 
[0027] System 100 includes a processor 102 and a 



data storage area (e.g., a memory) 104. Data storage 
area 104 includes certain well-known types of informa- 
tion, such as operating systems 112 and data and data 
structures 1 1 4. In the embodiment shown, data and data 
s structures 1 1 4 include, for example, web pages capabl 
of being displayed by a browser 106. Data and data 
structures 1 1 4 can also include, for example, applet soft- 
ware capable of being executed by a Java virtual ma- 
chine (JVM) 108. Storage area 104 preferably also in- 
fo eludes software (not shown) for communicating with a 
network connection 103, which can be a LAN, WAN, in- 
tranet, or the internet. 

[0028] A person of ordinary skill in the art will under- 
stand that system 100 may also contain additional ele- 
is ments, such as input/output lines; additional or second 
input devices, such as a keyboard, a mouse, and a voice 
input device; and display devices, such as a display ter- 
minal. System 100 may also include a computer reada- 
ble input device (not shown), such as a floppy disk drive, 
CD ROM reader, a chip connector, a chip connector, or 
a DVD reader, that reads computer instructions stored 
on a computer readable medium, such as a floppy disk, 
a CD ROM, a memory chip, or a DVD disk. System 100 
also may include application programs, operating sys- 
tems, data, etc., which are not shown in the figure for 
the sake of clarity. 

[0029] In the following discussion, it will be under- 
stood that the steps of methods and flow charts herein 
discussed herein can be performed by processor 102 
(or another appropriate processor) executing instruc- 
tions stored in storage area 104 (or other appropriate 
memories or storage areas). It will also be understood 
that the invention is not limited to any particular imple- 
mentation or programming technique and that the inven- 
tion may be implemented using any appropriate tech- 
niques for implementing the functionality described 
herein. The invention is not limited to any particular pro- 
gramming language or operating system. 
[0030] The instructions in storage area 1 04 may be 
read into storage area 104 from a computer-readable 
medium (not shown). Execution of sequences of instruc- 
tions contained in main memory causes processor 1 02 
(or a similar processor) to perform the processes de- 
scribed herein. In alternative embodiments, hard-wired 
circuitry may be used in place of or in combination with 
software instructions to implement the invention. Thus, 
embodiments of the invention are not limited to any spe- 
cific combination of hardware circuitry and software. 
[0031] Fig. 1 (a) shows a virtual machine (such as a 
Java virtual machine) 1 08 executing on system 1 00. Fig. 
1 (b) is a block diagram of a virtual machine that is sup- 
ported by system 100 of Fig. 1(a), and is suitable for 
implementing the present invention. When a computer 
program, e.g., a computer program written in the Java 
programming language, is executed, source code 160 
is provided to a compiler 170 within compiletime envi- 
ronment 155. Compiler 170 translates source code 160 
into bytecodes 180. In general, source code 160 is 
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translated into bytecodes 180 at the time source code 
160 is created by a software developer. 
[0032] Bytecodes 180 may generally be reproduced, 
downloaded, or otherwise distributed through a net- 
work, .g., network 1 03 of Fig. 1 (a), or stored on a stor- 
age device such as primary storage 104 of Fig. 1(a). In 
the described embodiment, bytecodes 180 are platform 
independent. That is, bytecodes 180 may be executed 
on substantially any computer system that is running on 
a suitable virtual machine. 

[0033] Bytecodes 1 80 are provided to a runtime envi- 
ronment 108, which includes virtual machine 190. Runt- 
ime environment 108 may generally be executed using 
a processor or processors such as processor 1 02 of Fig. 
1(a). Virtual machine 109 includes a compiler 192, an 
interpreter 194, and a runtime system 196. Bytecodes 
180 may be provided either to compiler 192 or to inter- 
preter 194. 

[0034] When bytecodes 1 80 are provided to compiler 
1 92, methods contained in bytecodes 1 80 are compiled 
into machine instructions. In one embodiment, compiler 
192 is a just-in-time compiler, which delays the compi- 
lation of methods contained in bytecodes 180 until the 
methods are about to be executed. When bytecodes 
180 are provided to interpreter 194, bytecodes 180 are 
read into interpreter 194 one bytecode at a time. Inter- 
preter 1 94 then performs the operation defined by each 
bytecode as each bytecode is read into interpreter 194. 
That is, interpreter 194 "interprets" bytecodes 180, as 
will be appreciated by those skilled in the art. 
[0035] When a method is invoked by another method, 
or is invoked from runtime environment 108, if the meth- 
od is interpreted, runtime system 1 96 may obtain the 
method from runtime environment 1 08 in the form of se- 
quences of bytecodes 180, which may be directly exe- 
cuted by interpreter 1 94. If, on the other hand, the meth- 
od that is invoked is a compiled method that has not 
been compiled, runtime system 108 also obtains the 
method from runtime environment 108 in the form of a 
sequence of bytecodes 108, which then may go to ac- 
tivate compiler 192. Compiler 194 then generates ma- 
chine instructions from bytecodes 180, and the resulting 
machine-language instructions may be executed direct- 
ly by processor 102 (or other appropriate processors). 
In general, the machine-language instructions are dis- 
carded when virtual machine 109 terminates. The oper- 
ation of virtual machines or, more particularly, Java vir- 
tual machines, is described in more detail in The Java 
Virtual Machine Specification by Time Lindholm and 
Frank Yellin (ISBN 0-201 -63452-X), which is incorporat- 
ed herein by reference. 

[0036] Fig. 2 shows an example of a set top box sys- 
tem, including a set top box 101 connected to a televi- 
sion 130 and a remote unit 120. Remote unit 120, which 
is used as an input device, is not shown to scale. An 
actual remote unit is generally of a size appropriate to 
fit in a user's hand. Similarly, set top box 101 and tele- 
vision 120 are not necessarily shown to scale. Certain 



embodiments of the present invention can also be used 
with a keyboard or with an infrared or other wireless key- 
board (not shown). 

[0037] It will be understood that the button arrange- 
5 ment shown on remote unit 1 20 are for purposes of ex- 
ample only and are not to be interpreted in a limiting 
sense. Remote unit 1 20 includes a "web" button, a "mail 
button, a "home" button, a "goto" button, a f ave" button, 
a "back" button, a "ctl" button, arrow buttons, and a 'se- 
10 lecf button. As will be appreciated, a conventional use 
of back button is to display a previously displayed web 
page. Similarly, a conventional use of "goto" button is to 
allow the user to specify and view a new web page. The 
arrow buttons allow the user to move a cursor between 
15 various elements on web page 304, as is known to per- 
sons skilled in the art. 

[0038] In Fig. 2, a button 204, a list and a panel are 
displayed on a web page displayed on display device 
130. It will be understood that these elements 304 are 

20 generated on a web page under control of an applet (not 
shown), such as applet 110 of Fig. 1 (a). 
[0039] Network connection 1 03 allows the set top box 
to communicate with a network, such as the internet, an 
intranet, a LAN or a WAN. Network connection can also 

26 be a wireless link. 

[0040] Video line 207 provides a source of video input 
to set top box 101. Line 206 couples the set top box 101 
to the display device 1 30 to convey video and audio out- 
put signals to the display device 1 30. The signals on line 

30 206 could be, for example, one or more of, S-video sig- 
nals, composite signals, component signals, or audio 
signals, HDTV signals. 

II. Detailed Discussion 

35 

[0041 ] Fig. 3(a) is a block diagram of parent and child 
context data structures in accordance with an embodi- 
ment of the present invention. In Fig. 3(a), a browser 
page memory 31 2 includes memory for a first applet ar- 

40 ea (which contains a list, a button, and a panel) and a 
second applet area (contents not shown). The browser 
memory includes a pointer to a parent context data 
structure 314 for the browser, which includes informa- 
tion about the browser, such as font, foreground and 

45 background color information, and screen coordinates 
of the browser. The parent context 314 has an associ- 
ated parent memory buffer 318, which includes visual 
representations of the screen memory for the browser. 
[0042] Each element in the HTML for a page currently 

so being displayed by the browser has an associated 
browser element 316. Thus, in the example, each of the 
first applet and the second applet have an associated 
browser element 316. Other types of elements that may 
have an associated browser element include GIFs, 

ss JPEGs, etc. 

[0043] Each browser element points to a child context 
302 in the JVM. Only one child context 302 is shown in 
the figure for the sake of clarity, but in the described em- 
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bodiment, each browser element (and therefore each 
applet) has an associated child context. The child con- 
text 302 includes information about its associated ap- 
plet, such as font, foreground and background color in- 
formation, and screen coordinates of information dis- 
played by the applet. The child context of an applet also 
includes a damage flag 350, which indicates whether 
the contents of the child buffer have been changed (akin 
to a "dirty bit) and a Recursive Update flag 352, which 
indicates whether the applet contains a reactive ele- 
ment, such as a button, list, or choice. The child context 
314 has an associated child memory buffer 304, which 
includes visual representations of the screen memory 
for the browser. 

[0044] The described embodiment does not use a 
graphics system such as an X Server or Motif. Instead, 
it uses direct graphic routines implemented along with 
child and parent contexts. These direct graphics rou- 
tines write directly into the graphics buffers of the parent 
and/or child contexts. A graphics Java object 31 0 points 
to a graphics peer object 308, which is an abstract class. 
A DWL object is a subclass of class 308. An example of 
the DWL graphics object could be a button widget. The 
DWL graphics object points to the child context 302 for 
the applet that contains the object. 
[0045] It is important to note that each Java applet ex- 
ecutes as one or more separate threads in the JVM. 
Thus, each Java applet can modify its child context 302 
and its child memory independently from the other ap- 
plets. As shown in Fig. 6, a main browser loop periodi- 
cally polls the executing applets to determine whether 
any of their damage flags are set. 
[0046] Fig. 3(b) is a diagram showing the relation- 
ships between a browser, exemplary browser elements, 
and exemplary components of a specific browser ele- 
ment. A browser displays HTML elements such as ap- 
plets, GIFs, JPEGs and frames. Certain elements, such 
as applets can include graphics components such as 
buttons, lists, choices, or panels. As shown, certain el- 
ements such as buttons, lists, and choices are examples 
of reactive components that require an interactive out- 
put when they are pressed or selected by the user. In 
contrast, panels generally do not require any external 
indication that the user has used them. 
[0047] Fig. 4 is a block diagram of an orphan data 
structure in accordance with ah embodiment of the 
present invention. Orphan contexts are used in situa- 
tions where some "behind the scenes" drawing needs 
tot be done. An orphan context is similar to a child con- 
text (and can be a special case of a child context) but is 
not associated with any particular applet or other ele- 
ment. Instead, as shown in Fig. 1 3, the content of an 
orphan context is transferred to a child buffer and the 
damage flag of the child buffer set so that the main 
browser loop will eventually pick up the graphics 
change. Orphan contexts cannot be drawn directly on 
the screen. They can only be copied to another context 
(such as a child context). 



[0048] Fig. 5 is a block diagram showing a main 
browser loop and a JVM loop. As is shown in the Figure, 
these loops include a main browser loop (which is per- 
formed by browser 106 of Fig. 1 ). A main JVM loop 504 

5 is performed by JVM 108. A key event queue 510 re- 
ceives "key events," for example, key presses from re- 
mote unit 206 and queues them for input to main brows- 
er loop 502. A JVM queue 508 receives "JVM events," 
from the main browser loop and queues them for input 

io to main JVM loop 504. In the described embodiment, 
each of loops 502and 506 are implemented as a thread 
executed by processor 102. 

[0049] Fig. 6 is a flowchart showing details of the main 
browser loop, including periodic refreshing of executing 

75 applets. In step 602, the main loop receives a key press 
from a key event queue. A key press can be the result 
of the user pressing a key on the remote 1 20 or the result 
of clicking or selecting a button or similar component dis- 
played on the screen by an applet. If in step 602, the 

20 current key event is an event for an applet, an event is 
placed on the JVM queue 508. Otherwise, an appropri- 
ate activity is performed for the key event. In either case, 
the main browser loop refreshes the graphics display of 
any executing applets, as shown in Fig. 7. (It will be ap- 

2S predated that other activities occur in the main browser 
loop, but have been omitted from the description for the 
sake of clarity). 

[0050] Fig. 7 is a flow chart showing details of refresh- 
ing the applets in the main browser loop. In an initial step 

30 702, control waits until the parent context is unlocked. 
In some situations, the parent context will already be un- 
locked. Locking of the parent context occurs when some 
other thread needs to access the parent context or 
wants to keep the parent context from being accessed. 

35 in the described embodiment, this lock is implemented 
as a unix mutex lock. One example of how the parent 
might become locked is shown in step 1002 of Fig. 10. 
[0051] The steps of Fig. 7 loop through the active ap- 
plets to determine whether its damage flag is set. If, in 

40 step 704, the damage flag of a current applet is set, con- 
trol passes to step 706. Otherwise, control passes to 
step 71 2. In step 706, the browser locks the parent and 
child contexts (so not other threads can change them 
during steps 708-710). In step 708, the browser "pulls" 

45 graphics data from the child context 302 and child mem- 
ory 304 of the applet to the parent context 31 4 and par- 
ent memory 318 of the browser. In the described em- 
bodiment, this step is performed via a call to a routine 
called aglPutContext, which is described in connection 

so with Fig. 14. 

[0052] Fig. 8 is a flow chart showing a main JVM loop 
for regular non- reactive components, such as a panel, 
or other display element that does not require an inter- 
active behavior. In the described embodiment, the steps 

55 of Fig. 8 are performed by the JVM 1 08. In step 802, the 
JVM receives an event from the JVM event queue. 
Events in this queue can be sent from the browser or 
from some other source. An example of such a non -re - 
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active event might be "draw a panel." Steps 804-81 2 lo- 
cate a Java AWT widget that has focus (i.e., that has 
been chosen to receiv the input from remote 1 20 or a 
similar keyboard). The event is delivered to a Java widg- 
et, which invokes the abstract event handler code and 
the actual component code for the peer. Step 812 lo- 
cates the DWL component widget corresponding to the 
peer, and, in step 814, the event is delivered to the DWL 
(Developer's widget library) component (see Fig. 14). In 
step 816, the DWL component draws the graphic in the 
child buffer in accordance with the event. This step is 
described in more detail in Fig. 10, which describes a 
method used for both reactive and non-reactive compo- 
nents. 

[0053] Fig. 9 is a flow chart showing a main JVM loop 
for reactive components, such as a button press. As dis- 
cussed above, a reactive component cannot wait for the 
main browser loop. In the described embodiment, the 
main browser loop of Fig. 7 executes every 200 millisec- 
onds. Similarly, the described reactive events, such as 
a button press are required to take place in approximate- 
ly 50 milliseconds. The steps of Fig. 9 are similar to 
those of Fig. 8, except that, in step 918, the JVM twice 
writes directly into the child and the parent contexts. 
Thus, the JVM directly writes the "pressed" button 
graphic and the "released" button graphic without wait- 
ing for the main browser loop. Step 918 sets the Recur- 
sive Update flag, draws two graphics images into the 
child and parent buffers, and clears the Recursive Up- 
date flag. This is described in more detail in connection 
with Fig. 10. 

[0054] Fig. 10 shows further details of Figs. 8 and 9. 
If the method of Fig. 10 is called from Fig. 8 (a non-re- 
active component), the Recursive Update flag will not 
be set. If, on the other hand, the method of Fig. 1 0 is 
called from Fig. 9 (a reactive component), the Recursive 
Update flag will have been set in step 918. 
[0055] Thus, for a non-reactive component, in step 
1002 the widget issues an aglBeginPaint call (see Fig. 
14), which sets the damage flag in the child context. Be- 
cause the recursive update flag is not set, control pass- 
es to step 1004. In step 1004, the DWL component ob- 
ject draws a graphics for the received JVM event in the 
child buffer. Because the recursive update flag is not set, 
control passes to step 1006 and then to step 1008. Thus, 
for a non-reactive component, the component object 
draws the correct graphic in the child buffer and sets the 
damage flag in the child context. The change in graphics 
will be picked up in the main browser loop as shown in 
Fig. 7. 

[0056] For a reactive component, in step 1002 the 
widget issues an aglBeginPaint call (see Fig. 14), which 
sets the damage flag in the child context (even though 
this is not required, it is harmless). Because the recur- 
sive update flag is set, a lock is also paced on the parent 
context and buffer and on the child context and buffer. 
Such a lock means that only the locking thread can 
change the contexts until they are unlocked. Control 



passes to step 1004. In step 1004, the DWL component 
object draws a first graphic for the received JVM event 
in the child buffer (such as a "pressed" button shown in 
Fig. 1 2(a)). Because the recursive update flag is set, the 
s DWL component object also draws the graphics for the 
received JVM event in the parent buffer. Thus, the 
change to the parent buffer will automatically be reflect- 
ed on the display screen without having to wait for the 
main browser loop. Control passes to step 1006 and 
10 then to step 1009, which clears the locks. Step 1010 
waits some approximate time, such as 50 milliseconds, 
which is long enough for a human user to notice that the 
button appearance has changed. 
[0057] Steps 1012-1016performsimilarstepstodraw 
is a second graphic (such as a "released" button shown in 
Fig. 12(b)) directly into the parent and child buffers and 
contexts. Thus, for a reactive component, the compo- 
nent object draws a first graphic in the child and parent 
buffers, waits some predetermined period of time, and 
draws a second graphics in the parent and child buffers. 
Reactive graphics do not wait for the change in graphics 
will be picked up in the main browser loop as shown in 
Fig. 7. If the main browser loop occasionally happens to 
redraw the second graphic, no harm is done. 
[0058] Fig. 11 is a diagram of the architecture of one 
embodiment of the present invention. It includes the ex- 
tended agl and DWL routines mentioned above. 
[0059] Fig. 1 3 is a flow chart for orphan context. The 
steps are similar to those of Fig. 8, but in step 1 31 6, the 
orphan context must be copied to a child context (or a 
parent context, not shown) where it will eventually be 
displayed. Orphan context are used, for example, when 
there is a need to draw a graphic "offscreen," such as 
sometimes occurs for JPEGs and GIFs. Similarly, if a 
graphic will be duplicated once it is drawn, it is often ad- 
vantageous to use an orphan context. 
[0060] Fig. 14 shows exemplary agl (applications 
graphics library) routines used to implement child and 
orphan contexts in an embodiment of the present inven- 
tion. These routines include but are not limited to: 
aglCreateChildContext, aglDeleteChildContext, aglBe- 
ginPaint, aglEndPaint, aglCreateOrphanContext, 
aglPutContext, and aglContextRedrawn. 
[0061] In summary, the described embodiment of the 
present inventbn provides one or more child contexts 
that correspond to elements in the HTML for a web page 
displayed by a browser. For example, each applet exe- 
cuted by the browser has a corresponding and separate 
child context. Each child context has an associated 
memory buffer. The browser also has a parent context, 
which each child context points to. When a graphic is 
displayed via a widget, the widget draws the graphic 
(such as a panel or a non-pressed button) in the child 
context of the applet and sets a "damage" flag in the 
child context. When the browser performs its main 
browser loop, it checks the status of the damaged flag 
for each element (including each applet). If the browser 
finds a damaged flag that is set, it "pulls" the information 



25 



30 



35 



40 



45 



50 



6 



11 



EP 1 006 440 A2 



12 



from the child buffer into the parent buffer, which is used 
to update the display screen. Thus, several separate 
thr ads in th JVM can be executing concurrently and 
updating graphics elements without interfering with 
each other. 

[0062] The invention extends the AGL (Applications 
graphics library) routines to include (at least) routines to 
set up, delete, read, and modify child contexts and to 
copy information from child contexts to parent contexts. 
The DWL (developer's widget library) routines are ex- 
tended to include a parameter identifying the context 
currently being worked on. The browser has been ex- 
tended to implement the functionality of checking the 
damage flag of the context for each applet and pulling 
information from a buffer of a child context into the buffer 
of the parent context. 

[0063] Reactive components, such as buttons, lists, 
choice bars, etc. present special problems. When a 
widget, for example, a button, needs to show interactive 
behavior when pressed (like going up and down), this 
behavior must occur very rapidly - too rapidly to neces- 
sarily be picked up by a main browser loop, which exe- 
cutes relatively infrequently. Thus, a child context can- 
not update the display properly for a reactive component 
if it merely marks the child context as damaged and 
waits for the browser loop. 

[0064] While the invention has been described in con- 
junction with a specific embodiment, it is evident that 
many alternatives, modifications and variations will be 
apparent to those skilled in the art in light of the forego- 
ing description. Accordingly, it is intended to embrace 
all such alternatives, modifications and variations as fall 
within the spirit and scope of the appended claims and 
equivalents. 



Claims 



returning execution control to the browser from 
the virtual execution environment. 

2. The method of claim 1 , further comprising providing 
5 a second child context in the virtual execution envi- 
ronment, corresponding to the parent context in the 
browser, where the child context corresponds to a 
second applet that is executed by the browser. 

10 3. An apparatus that displays a reactive graphics ele- 
ment, comprising: 

a browser having a parent context used to up- 
date a display, the browser being configured to 
send a reactive graphics event to a virtual ex- 
ecution environment; 

software circuitry configured to provide, in the 
virtual execution environment, a child context 
corresponding to the parent context and an ap- 
plet that is executed by the browser; 
software circuitry configured to write, by the vir- 
tual execution environment, a first graphic into 
a buffer of the child context and a buffer of the 
parent context, in accordance with the reactive 
graphics event, which will be displayed on the 
display ; 

software circuitry configured to wait a predeter- 
mined amount of time; 

software circuitry configured to write, by the vir- 
tual execution environment, a second graphic 
into the buffer of the child context and the buffer 
of the parent context, in accordance with the 
reactive graphics event, which will be displayed 
on the display; and 

software circuitry configured to return execu- 
tion control to the browser from the virtual exe- 
cution environment. 
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1 . A method of displaying a reactive graphics element, 
comprising: 4( > 

sending a reactive graphics event, by a browser 
to a virtual execution environment; 
providing, in the virtual execution environment, 
a child context corresponding to a parent con- 45 
text in the browserand an applet that is execut- 
ed by the browser; 

writing, by the virtual execution environment, a 
first graphic into a buffer of the child context and 
into a buffer of the parent context, in accord- so 
ance with the reactive graphics event, which 
will be displayed on a display screen; i 
waiting a predetermined amount of time; 
writing, by the virtual execution environment, a 
second graphic into the buffer of the child con- ss 
text and the buffer of the parent context, in ac- 
cordance with the reactive graphics event, 
which will be displayed on a display screen; and 



4. A computer program product comprising: 

a computer usable medium having computer 
readable code embodied therein for causing a 
data processing system to display a reactive 
graphics element, including: 
computer readable program code configured to 
cause the data processing system to send a re- 
active graphics event, by a browser to a virtual 
execution environment; 

computer readable program code configured to 
cause the data processing system to provide a 
child context in the virtual execution environ- 
ment, the child context being associated with a 
parent context in the browser, where the child 
context corresponds to an applet executed by 
the browser; 

computer readable program code config- 
ured to cause the data processing system 
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to write, by the virtual execution environ- 
ment, a first graphic into a buffer of the child 
cont xt and a buffer of the parent context, 
in accordance with the reactive graphics 
event, which will be displayed on a display 
screen; 

computer readable program code config- 
ured to cause the data processing system 
to wait a predetermined amount of time; 
computer readable program code config- 
ured to cause the data processing system 
to write, by the virtual execution environ- 
ment, a second graphic into the buffer of 
the child context and the buffer of the par- 
ent context, in accordance with the reactive 
graphics event, which will be displayed on 
a display screen; and 
computer readable program code config- 
ured to cause the data processing system 
to return execution control to the browser 
from the virtual execution environment. 

5. The method of claim 1 , wherein the virtual execution 
environment provides applications graphics library 
(AGL) routines, including a child context set-up rou- 
tine, a child context delete routine, a child context 
read routine, a child context modify routine, and a 
routine for copying information from the child con- 
text to the parent context. 

6. The method of claim 5, wherein the method further 
includes invoking a routine, the routine being one 
of the AGL routines. 

7. The method of claim 1 , wherein the applet is one of 
multiple applets executed as different threads by 
the browser, each of the applets being associated 
with a child context, each child context correspond- 
ing to the parent context. 

8. The method of claim 7, wherein each child context 
corresponds to a browser element displayed on the 
display. 

9. The method of claim 1 , wherein the child context 
further includes a damage flag indicating whether 
the contents of a corresponding child buffer has 
changed. 

10. The method of claim 1, wherein the child context 
further includes a recursive update flag indicating 
whether the associated applet contains a reactive 
element. 



1 1 . A method of graphic widget interaction, comprising: 55 

receiving a key event indicating one of a press, 
release and selection, the key event being ei- 
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ther reactive or non-reactive, the selection in- 
dicating selection of an item displayed by an ap- 
plet that is ex cuted by a browser, the browser 
having a parent context that is used for updat- 
ing a display; 

providing, in a virtual machine, a child context 
associated with the parent context; and 
performing, by the virtual machine, a virtual ma- 
chine loop, including: 

locating a graphic widget corresponding to 
the key event; 

setting a recursive update flag in the child 
context if the key event is reactive; 
resetting the recursive update flag in the 
child context if the key event is non-reac- 
tive; 

setting a damage flag in the child context; 
using the corresponding graphic widget to 
update the child context alone if the recur- 
sive update flag is reset; and 
updating both the child context and the par- 
ent context, via the corresponding graphic 
widget, if the recursive updating flag is set, 
wherein the updating of the parent context 
is reflected automatically on the display 
without having to wait for a browser loop. 

12. The method of claim 11, further including: 

placing the key events in a queue for input to a 
browser loop; and 

performing, by the browser, the browser loop, 
including; 

performing an activity if the key event is 
press or release; 

posting a virtual machine event in a queue 

for input to the virtual machine loop, if the 

key event is selection; and 

refreshing each applet that is executed by 

the browser, the refreshing for each applet 

including: 

using the damage flag for determining 
the need for refreshing the applet; 
locking an associated child context 
and the parent context; and 
pulling the graphics data from the as- 
sociated child context to the parent 
context for updating the display. 

13. The method of claim 12, wherein the key events of 
press and release indicate pressing and releasing, 
respectively, of one of a button and key 
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CED Consumer Device Architecture 




BCM: Browser Control Module 
MBA: Merrimack Browser A 
DWL: Developer's Widget Library 
ODL: Opaque Device Library 
DSL: Developer's System Library 
AGL: Applications Graphics Lab 
DFL: Developer Font Library , 
ODI: Opaque Device Implementation 
OSS: OS Specific 
PSS: Project Specific Services 



APL: Appliance Printer Library 
PML: Programmers Mail Library 
DFS: Developer File S 
PWL: Programmer's WEB Library 



Fig. 11 
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